In an effort to achieve further processing power for computing, many designers of computers are turning to parallel processing. There is, however, a wide variety of computer architectures which employ parallel processing.
Parallel processing computers can be divided into two general types: those which have shared-memory processors and those which have distributed-memory processors. Shared-memory computers involve multiple processors which can access the same memory. In contrast, distributed-memory computers have processors with private independent memories. Communication between distributed-memory processors is achieved by a communications interconnection. The present invention is specifically concerned with distributed-memory parallel processing computers, and more particularly to a toroidally-connected computer.
The performance of a toroidally-connected distributed-memory parallel computer is limited by several factors: the total memory of the computer, the number of processors in the computer, the communication between processors required by a given problem, and the speed at which data can be moved in and out of the computer. The first two factors are relatively insignificant since, in most parallel computers, the number of processors and their memory can be varied readily to suit whatever need exists. The third factor, that of processor intercommunication, is a topic of widespread and intensive research (see "M.sup.2 Mesh: An Augmented Mesh Architecture"; Lin & Moldovan; Proceedings of First International Conference on Parallel Processing, p. 306; and "The Mesh With A Global Mesh: a flexible high-speed organization for parallel computation"; Carlson; Proceedings of First International Conference on Supercomputing, p. 618.). For large data set applications (such as image processing or data base examination), the input/output (I/O) requirements may well be the critical factor constraining the overall throughput of the parallel computer. In this specification the term toroidal will be understood to include the term mesh. As shown in FIG. 2, there is a toroidal array of processors wherein each processor is connected to four neighboring processors.
There are two general parallel processing modes frequently used: distributed and pipelined. Distributed processing requires that every processor perform the same operation on a different subset of the data. Pipelined processing requires that every processor perform a different operation on the same set of data. Toroidally-connected computers are particularly suitable for operating in both modes but they still have problems.
Attempts to provide adequate I/O generally involve providing each processor node with an I/O channel and having a controller write sequentially through the I/O channel to the various processors. This approach lends itself to distributed processing, but is relatively expensive in terms of chip cost, and circuit board area, (see U.S. Pat. No. 4,514,807; "Parallel Computer"; Nogi; also the commercial products "Connection Machine" from Thinking Machines, "Hypercube" from Intel Scientific Computers, and the "Ncube/10" from Ncube.). Furthermore, the device feeding the I/O channels imposes still another bottleneck.
An alternative approach sometimes taken is to perform I/O only with one processor and use the interconnection network of the parallel processor to further distribute the data. This approach is inexpensive but suffers from bandwidth constraints.
Both of these general approaches are incapable of performing I/O with multiple data sources or destinations. This limits their usefulness when co-operating with other computers and data storage or generation devices. Neither of these approaches can be equally effective for both pipelined and distributed parallel processing. Neither of these approaches can offer flexible I/O rates. Since one of the prime advantages of a parallel system is its performance flexibility (adding processors to improve computational performance), it is important to scale the I/O performance and hardware in a similar fashion.
The object of this invention is to provide a more effective architecture for a toroidally-connected distributed-memory parallel computer which obviates the above-noted problems.